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Summary 


/A 



Maximum likelihood estimation of parameters in linear struc- 
tural relationships under normality assumptions requires knowledge 
of one or more of the model parameters if no replication is avail- 
able. The most common assumption added to the model definition is 
that the ratio of the error variances of the response and predictor 

I 

variates is known. This article investigates the use of asymptotic 
formulae for variances and mean squared errors as a function of 
sample size and the assumed value for the error variance ratio. 


Some key words: Errors in variables; Identif lability ; Regression 


1. Introduction 

Linear structural relationships are linear models between two 
stochastic variates (Y,X) in which both variates are measured with 
error. Let = a + SX^ and define observable variates 

X 1 ” X i + u i y i = Y i + v i’ 1 " 1 » 2 »***» n * (1.1) 

■ 2 2 2 

Assume further that independently X^N(p v ,a v ), u^N(0,cf ) t and v^N(0,cr ). 

A A tl V 
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Under these model assumptions it is well known that in the absence 
of replication no consistent estimators of a and 8 exist because the 
model lacks Identifiability (e.g., Madansky 1959). Geary (1942) 
showed that when (u,v) are jointly normally distributed. If X 
possesses a finite cumulant of order greater than two then 3 is 
Identifiable in the joint distribution of (x,y)j thus, nonnormal 
distributions for X generally allow Consistent estimation of 8. 
Reiersol (1950) strengthened this result by proving that if (1) u 
and v are independently distributed or (ii) (u,v) is bivariate 
normal, nonnormality of X is a necessary and sufficient condition 
for identifiability of 8. Reiersol ’s results are summarized in 
Table 1. Note especially that a is identifiable (and estimable) 
when 8 is identifiable; consequently, the focus of this article is 
on the estimation of the slope parameter 8. 

[Insert Table 1] 

The identifiability conditions displayed in Table 1 pertain 

to linear structural models in which none of the model parameters 

are known. Kendall and Stuart (1977, ^Chapter 29) detail various 

solutions to the likelihood equations when one or more of the 

variances in model (1.1) are known. Of important theoretical 

2 2 

interest is the assumption that \ = o^/c^, the ratio of error 
variances, is known. Under this assumption the joint distribution 
of (y,x) is Identifiable and the likelihood equations have a unique 
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solution for Pi 

5 « s(A) + fi(s Hs 2 (A) + AJ 1/2 , s(A)*»(s 2 “As 2 )/ (2s ) (1.2) 

A/ y A *v 

2 2 

where s , s , and & are the sample variances and covariance, 
y x xy 

respectively, and C(s )-sign(s ). This solution is consistent 

xy xy 

and asymptotically normal and all estimators of the model variances 
are assured to be nonnegative. While this estimator is commonly used 
to estimate the slope parameter, few theoretical or simulation studies 
have been conducted to evaluate the adequacy of asymptotic variance 
formulae for finite sample sizes or the sensitivity of estimator (1.2) 
to erroneous selection of the variance ratio A. 

In this article both of the above topics are investigated. 

Section 2 contains asymptotic variance and mean squared error formulae 
for estimator (1,2) for both correct and incorrect choicer 'f A. 
Section 3 presents the results of a simulation study in which the 
sample size and the assumed value of A are varied for several model 
configurations. Replication of observations is discussed in Section 
4 and concluding remarks are made in Section 5. 

2. Asymptotic Properties 

Asymptotically (l.e., replacing sample moments by their para- 
meter values) , 

90/aA = -gt/(g 2 +A) , (2.1) 

2 2 

where t = cr /cr v is a "noise-to-signal ratio" for the observable pre- 

U A 

A 

dictor variable x. The rate of change of g with respect to A is 
thus seen to depend on the true values of g, A, and t. Figure 1 
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Illustrates the general features of equation (2*1)? 0 is relatively 

insensitive to the true value of X for large values of X and small 

o 

values of t. Together these two conditions imply that cr^, the error 

variance for the observable variate x, is small. In other words, 

under the conditions for which the linear structural model (1.1) is 

usually proposed (i.e., t moderate to large or X small to moderate — 

2 

each implying that is nonnegligible) , the estimator (1.2) can be 
very sensitive to the true value of X. 

[Insert Figure Ij 

A similar perspective on the sensitivity of (1,2) to the value 

of X is obtained by assuming X is stochastic rather than constant. 

Llndlsy and El-Sayyad (1968) suggest assuming a uniform (k~\k) prior 

for X if the two measurement errors are believed to be of the same 

2 

magnitude. Alternative proposals might include N(k,a^) or Chisquare(k) 
priors. Using statistical differentials (e.g., Serfling 1980) one 
can approximate the expectation of (21) using a three- term Taylor 

A 

series expansion of 33/3X. The approximate expectations are, 
respectively, 

~2pt[(26 2 + k + k -1 )" 1 + (k-k“ 1 ) 2 {3 (2(3 2 + k + k -1 ) 3 }” 1 ] (2.2) 

-3t[(f3 2 + k)" 1 + o 2 (B 2 + k)“ 3 ] (2.3) 

-3t[(3 2 + k)" 1 + 2k(3 2 + k)" 3 ] . (2.4) 

Graphs of equations (2.2) to (2.4) as a function of k are variants 
of Figure 1, all resulting in the same general conclusion: the 
slope estimator (1.2) is relatively insensitive to the true value 
of X only when t is close to zero and k is large. 
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The slope estimator (1»2) is asymptotically unbiased when the 
error variance ratio is known. Again applying the method of statis- 
tical differentials, the asymptotic variance of (1.2) (Ignoring terms 
of 0(n“ 2 )) is 

n~ 1 [(3 2 + X)t + It 2 ] , (2.5) 

which reduces to equation (9) of Robertson (1974) when X ■ 1. For 

comparative purposes, the asymptotic mean squared error for the least 

2 

squares estimator (s /s ) of the slope parameter under the assumptions 

xy x 

accompanying model (1.1) is (c.f., Richardson and Wu 1970, equations 
(2.24) and (2.25)) 

0 2 t (n"^ + t)(l + t)” 2 + n^XtCl + t)" 1 . (2.6) 

Note that the least squares estimator is the maximum likelihood esti- 

2 2 2 

mator when a = 0, in which case (2.6) reduces to a /(nc ). 

u V A 

The foregoing expressions enable one to assess the sensitivity 

of the linear structural estimator (1.2) to the true value of the 

ratio of error variances. In application it is also of interest to 

examine the sensitivity of (1.2) to an erroneous choice of X. When 

X is incorrectly specified, (1.2) is no longer asymptotically unbiased. 

-2 

Ignoring terms of 0(n ), the asymptotic expectation and variance of 

(1.2) using an assumed value X* for the ratio of error variances are, 
respectively, 

E(P) « + <5[g A (X*)][g 2 (X*) + X*] 1/2 (2.7) 

var (3) = n“ 1 (e 2 +X*)'” 2 [36 2 t 2 (X-X*) 2 +(3 2 +X*) 2 ((6 2 +X)t+Xt 2 }] (2.8) 

where g x (X*) * [(8 2 -X*)a 2 +(X-X*)a 2 ]/[23a 2 ] and 6 [g x (X*) ]=sign[g x (X*) ] . 

A 

When X* = X, bias(6) = 0 and equation (2.8) reduces to (2.5). 


6 


Figures 2 and 3 compare the asymptotic mean squared errors of 

the structural model estimator (1.2) with the least squares esUl~ 

mator, the letter mean squared error calculated from equation (2.6). 

2 

In Figure 2 the true variance ratio X is assumed known and 6, V 
2 

and o v are fixed at 3, 5, and 10, respectively. Unless X is extremely 
small, corresponding to relatively small error in the response vari- 
able, the structural model estimator has a smaller asymptotic mean 
squared error than least squares, with the Improvement offered by 
the structural model estimator Increasing with the sample size and 
decreasing with X. 

[Insert Figure 2] 

If X* is chosen incorrectly, Figure 3 demonstrates that the 

benefits of using structural model estimators over least squares 

diminishes as X* differs from X. For this figure the model para- 

2 

meters are set at (6, u , X) = (3, 5, 6). Both Figures 2 and 3 
are illustrative of a general conclusion which can be drawn from 
a comparison of the asymptotic mean squared errors: X* must be In 
relative proximity to the true value X for the structural model 
estimator to be a substantial improvement over least squares. 

[Insert Figure 3] 

3. Simulation Results 

In each of Tables 2 to 5, 1000 replications of samples of size 
n were generated from model (1.1) with normal variates generated by 
I.M.S.L. subroutine GGNML on a C.D.C. 6600 computer. Table 2 compares 
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A 

the average 0 calculated using equation (l c 2) with the true value of 

2 ? 

0. Xu this table 0, cr vS and o ’ are fixed at 3* 5 S and 5 9 respectively* 

A Ll 

2 

so that varying o v the results are only a function of X and n. For 
samples of size 50 and 100 the maximum relative error in estimating 
0 usi^g the correct value of X* is 4%# Incorrectly choosing X* larger 
than the true variance ratio results in underestimation of 0 whereas 
too small a selection of X* restilts in overestimation of 0, 

[Insert Table 2] 

Estimated and asymptotic mean squared errors are compared In 
Table 3. Estimated mean squared errors are computed from the usual 
formula, 

mse s E (0=0)^/1000 ? 

and asymptotic mean squared errors are obtained froia equation (2.5) 

A 

(recall that 0 is asymptotically unbiased when X is known) using the 
true values of 0, X, and t. The ratios in Table 3 corresponding to 
correct assumed values of X* indicate that use of asymptotic formulae 
for moments of structural model estimators cannot be recommended for 
samples of size 100 or less. Even when X* is chosen correctly and 
the true model parameters are inserted in the asymptotic formulae, 
samples of size 100 result in errors of 15-30% between sample mean 
squared errors and those calculated from equation (2.5). 

[Insert Table 3] 

Tables 4 and 5 display ratios of sample and asymptotic mean 
squared errors for samples of size 200 for a variety of values of 
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3i X, and t„ Whan X* Is chosen correctly the ratios are much 
closer to 1.0 in these tables than in Table 3, If a relative 
error of approximately 10% or less is acceptable, samples of size 
200 could be considered minimally acceptable for a wide range of 
model parameters. 

[Insert Tables 4 and 5] 

Tables 3 to 5 also demonstrate that X* must be selected near 
its true value for the asymptotic variance formula (2.5) to provide 

A 

a reasonable assessment of the variability of 3. When 3 is small 
it is especially undesirable to choose values of X* which are less 
than the true ones. The deleterious effects of erroneous sanction 
of X* decrease with larger values of 3 and smaller values of t. 


4. Replication 


Replication of observations for one or more specific values of 
X allows consistent estimation of 3 when X Is unknown. Dorff and 
Gurland (1961) investigate four analysis of variance estimators of 3 
in functional equation models (the are assumed to be unknown 

constants) when, one or more of the are replicated. On the basis 

of asymptotic mean squared error comparisons when an equal number of 
replications is available for each X^ they prefer an estimator 
similar to equation (1.2) in which the following estimator of X 
is inserted in place of the true value: 


w /w 
yy xx 

n r 


n 


z z (y-.-y. ) 2 / z z (x-.-x. ) 2 , 
=l j=l i-i j=i 


(4.1) 
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where r replicates for both x and y axe available fox each of the 

n X^ Under the structural model assumptions (1.1) this estimator 
* 

of 0 Is consistent and has asymptotic variance equal to 

m<0 2 +A)+Anm 2 (0 2 ~A) 2 (0 2 +Ar 2 + 4A 2 m 2 0 2 (0 2 -fA)“ 2 (rn/(r-l) ] (4.2) 

where m ■ t/rn. Equation (4.2) corresponds to var^Cb^) of Dorff 
and Gurland (1961) , 

Barnett (1970) derives the maximum likelihood estimator of 0 
for functional models. Dolby (1976) derives maximum likelihood 
estimators of 0 for an "ultras tructural" model which Includes the 
structural and functional models as special cases. When an equal 
number of replicates are available for each X^, he shows that the 
maximum likelihood estimator of 0 is again of the form (1.2) with 
the following estimator of A: 


A =(s w ~s w )/(s w ~s w ) , 
yy xy xy yy ' xx xy xy xx' ' 


(4.3) 


where s = r£(y. -y ) , etc. The two estimators of A, equations 

yy • • • 

(4.1) and (4.3) f , are asymptotically equivalent since plira(w xy ) = 0; 

A 

therefore, the asymptotic variance of 0 using (4.3) to estimate A 
is given by equation (4.2). Although the maximum likelihood esti- 
mator and the analysis of variance estimator are asymptotically 
equivalent, the latter estimator. might be preferable with small 
sample sizes since A estimated by (4.3) can be negative (see Dolby 


(1976)). 

Tables 6 and 7 compare empirical and asymptotic properties of 
the analysis of variance structural model estimator based on 
equation (4.1). The summary statistics displayed in these tables 
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are computed from 1000 simulated experiments with (0,0^,aJ)“(3>5*5) 
and r * 2 and S replicates. The analysis of variance estimator pro- 
duces satisfactory agreement {relative arror less than 10%) between 
the average estimate of 0 and its true value for sample sizes as 
small as n « 20 with r “ 2 replicates. Agreement between the empirical 
and asymptotic mean squared errors again requires a sample size of at 
least n « 200 (total sample size N ■ nr) for an empirical relative 
error of approximately 10% or less. 

[Insert Tables 6 and 7] 

Erroneous use of least squares when the predictor variable is 
measured with error is especially unwarranted when X can be esti- 
mated with replicated observations. Figure 4 illustrates that esti- 

A 

mator (1.2) using equation (4.1) for X substantially improves esti- 
mator accuracy over least squares, even for small sample sizes. The 
model parameters used in the construction of this figure are the same 
as those used in Figure 2. Maximum likelihood estimation of 0 using 
equation (4.3) results in simulation results comparable to Tables 6 
and 7 and mean squared error improvement over least squares equal to 
that displayed in Figure 4. 

[Insert Figure 4] 

5* Discussion 

The results presented in Sections 3 and 4 are only a portion 
of a larger study in which simulations and asymptotic comparisons 
were conducted for a wide range of model parameters. The tables 


I# 
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and figures are illustrative, of the overall results In general 
the effective use of asymptotic properties of the structural esti- 
mator (1.2) requires a large sample size and accurate selection of 
the variance ratio X when the variance ratio cannot be estimated 
from replicated observations. Correct selection of X and a large 
sample size also assures a smaller mean Bquared error than least 
squares unless the variance ratio is very small. Incorrect selec- 
tion of X, especially the selection of too small a value, compro- 
mises the effectiveness of the structural model estimator relative 
to least squares. Replication of observations for one or more 
specific values of X Is an effecting alternative to least squares 
when all model parameters are unknown provided that the total sample 
size is sufficiently large. 

The investigations reported in this paper assume that the true 
predictor variable is normally distributed. If X Is nonnormal all 
model parameters. Including X, are usually estimable. Unfortunately 
the derivation of maximum likelihood estimators Is theoretically 
intractable for many Important distributions; e.g., XMJeta(a,b) 
and (u,v) normally distributed . Although moment estimators of (3 
are available, they are not unique and are often inefficient. 

Alternatives to moment estimators are currently under investigation 
and will be reported in the near future. 
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Table 1, Ident.lfiabiXSi,/ conditions for linear structural models 


(a) Identlfiability of B 

(i) X is nonnonnally distributed and either u and v are 
Independent or (u,v) is bivariate normal 

(ii) X is normally distributed and the distribution of neither 
u nor v is divisible by a normal distribution 


(b) 3 is identifiable 

(i) a is identifiable 

(ii) if u and v are independent and the characteristic functions 
of u, v, and X are continuous, all other model parameters are 
identifiable 

(Hi) if (u,v) is normally distributed, all other model parameters 
are identifiable iff 

(1) the distribution of X (and Y) is not divisible by a 
normal distribution, and 

(2) either u=0 or v=0 


Table 2. Ratio of simulated and asymptotic expectations of structural model slope estimator 


Pi \ 
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Table 3. Ratio of simulated and asymptotic (using true A) mean squared errors of the 
structural model slope estimator 
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Table 4. Ratio of simulated and asymptotic (using true A) mean squared errors of the 
structural model slope estimator: n — 200, t = .1 
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Table 5. Ratio of simulated and asymptotic (using true A) mean squared errors of the 
structural model slope estimator: m = 200, t = 1 
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Fig. 2. Ratio of the asymptotic mean squared error of the 
structural model slope estimator and the least 
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